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We introduce the polynomial coefficient matrix and identify maximum rank of this 
matrix under variable substitution as a complexity measure for multivariate polyno- 
mials. We use our techniques to prove super-polynomial lower bounds against several 
classes of non-multilinear arithmetic circuits. In particular, we obtain the following 
u: results : 

u 
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• As our main result, we prove that any homogeneous depth-3 circuit for computing 
the product of d matrices of dimension n x n requires Q{n'^~^ /2'^) size. This 
improves the lower bounds in |9j when d = uj{l)- 

• There is an explicit polynomial on n variables and degree at most ^ for which 
QQ , any depth-3 circuit C of product dimension at most ^ (dimension of the space of 

I affine forms feeding into each product gate) requires size 2^("). This generalizes 

the lower bounds against diagonal circuits proved in p!4]. Diagonal circuits are 
^ ■ of product dimension 1. 

CD ■ • We prove a n^i^°^^) lower bound on the size of product-sparse formulas. By 

definition, any multilinear formula is a product-sparse formula. Thus, our re- 
sult extends the known super-polynomial lower bounds on the size of multilinear 
formulas [Hj. 

• We prove a 2^^"-' lower bound on the size of partitioned arithmetic branching 
programs. This result extends the known exponential lower bound on the size of 
ordered arithmetic branching programs [7j. 
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1 Introduction 



Arithmetic circuits is a fundamental model of computation for polynomials. Establishing 
the limitations of polynomial sized arithmetic circuits is a central open question in the area 
of algebraic complexity(see pj^j for a detailed survey). One of the surprises in the area was 
the result due to Agrawal and Vinay |2] where they show that if a polynomial in n variables 
of degree d (linear in n) can be computed by arithmetic circuits of size 2°^'^\ then it can 
be computed by depth-4 circuits of size 2°'^"'). The parameters of this result was further 
tightened by Koiran [8]. These results explained the elusiveness of proving lower bounds 
against even depth-4 circuits. For depth-3 circuits, the best known general result (over finite 
fields) is an exponential lower bound due to Grigoriev and Karpinski [5] and Grigoriev and 
Razborov [Ij. Lower bounds for restricted classes of depth-3 and depth-4 circuits are studied 

in [n m US] . 

One class of models which has been extensively studied is when the gates are restricted to 
compute multilinear polynomials. Super-polynomial lower bounds are known for the size of 
multilinear formulas computing the permanent/determinant polynomial [12j. However, even 
under this restriction proving super-polynomial lower bounds against arbitrary multilinear 
arithmetic circuits is an open problem (see [IT]). The parameter identified by [11], which 
showed the limitations of multilinear formulas, was the rank of a matrix associated with 
the circuit - namely the partial derivatives matrix0- The method showed that there exists a 
partition of variables into two sets such that the rank of the partial derivatives matrix of any 
polynomial computed by the model is upper bounded by a function of the size of the circuit. 
But there are explicit polynomials for which the rank of the partial derivatives matrix is 
high. This program has been carried out for several classes of multilinear polynomials and 
several variants of multilinear circuits[3l [TJ [TOl [TTl [121 [13] • However, this technique has 
inherent limitations when it comes to proving lower bounds against non-multilinear circuits 
because the partial derivatives matrix, in the form that was studied, can be considered only 
for multilinear circuits. 

In this work, we generalize this framework to prove lower bounds against certain classes 
of non-multilinear arithmetic circuits. This generalization also shows that the multilinearity 
restriction in the above proof strategy can possibly be eliminated from the circuit model 
side. Hence it can also be seen as an approach towards proving lower bounds against the 
general arithmetic circuits. 

We introduce a variant of the partial derivatives matrix where the entries will be poly- 
nomials instead of constants - which we call the polynomial coefficient matrix. Instead of 
rank of the partial derivatives matrix, we analyze the maxrank - the maximum rank of the 
polynomial coefficient matrixH under any substitution for the variables from the underlying 
field. We first prove how the maxrank changes under arithmetic operations. These tools are 

^An exponential sized matrix associated with the multilinear polynomial with respect to a partition of 
the variables into two sets. See Section [2] for the formal definition. 

^When it is clear from the context, we drop the matrix as well as the partition. By the term, maxrank 
of a polynomial, we denote the maximum rank of the polynomial coefficient matrix corresponding to the 
polynomial with respect to the partition in the context. 
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combined to prove upper bounds on maxrank of various restrictions of arithmetic circuits. 

In [H], it was proved that any homogeneous depth-3 circuit for multiplying dnxn matrices 
requires Q (n'^~^/d\^ size. We use our techniques to improve this result in terms of the lower 
bound. Our methods are completely different from |9] and this demonstrates the power 
of this method beyond the reach of the original partial derivatives matrix method due to 
Raz [11]. We prove the following as the main result of this paper. 

Theorem 1.1 (Main Result). Any homogeneous depth-3 circuit for computing the product 
of d matrices of dimension n x n requires Q{n'^~^ /2'^) size. 

Notice that compared to the bounds in [9], our bounds are stronger when d = uj{l). 
Very recently, Gupta et al[6] studied the model of homogeneous circuits and proved a strong 
lower bound parameterized by the bottom fan-in. They studied depth-4 circuits (SnSII) 
and showed that if the fan-in of the bottom level product gate of the circuits is t, then any 
homogeneous depth-4 circuit computing the permanent (and the determinant) must have 
size 2^^^\ In particular, this implies 2^*^") lower bound for any depth-3 homogeneous circuit 
computing the permanent (and the determinant) polynomial of n x n matrices (n^ variables). 
However, we remark that Theorem II. II is addressing the iterated matrix multiplication poly- 
nomial and hence is not directly subsumed by the above result. Moreover, the techniques 
used in [0] are substantially different from ours. 

We apply our method to depth-3 circuits where space of the affine forms feeding into 
each product gate in the circuit is of limited dimension. Formally, a depth-3 SHE circuit 
C is said to be of product dimension r if for each product gate P in C, where P = Uf^^Li, 
where Li is an affine form for each i, the dimension of the span of the set {Li}i^[d] is at most 
r. 

We prove exponential lower bounds on the size (in fact, the top fan in) of depth-3 circuits 
of bounded product dimension for computing an explicit function. 

Theorem 1.2. There is an explicit polynomial on n variables and degree at most | for which 
any SIIS circuit C of product dimension at most ^ requires size 2^*^"^ . 

In [H], the author studies diagonal circuits, which are depth-3 circuits where each product 
gate is an exponentiation gate. Clearly, such a product gate can be visualized as a product 
gate with the same affine form being fed into it multiple times. Thus, these circuits are 
of product dimension 1, and our lower bound result generalizes size lower bounds against 
diagonal circuits. 

Note that the product dimension of a depth-3 circuit is different from the dimension of 
the span of all affine forms computed at the bottom sum gates of a SIIS circuit. We will 
show that this parameter, which we refer to as the total dimension of the circuit, when 
bounded, makes the model non-universal. 

For our next result, we generalize the model of syntactic multilinear formulas to product- 
sparse formulas (see section [2] for a definition). These formulas can compute non-multilinear 
polynomials as well. We show the following theorem regarding this model using our methods. 
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Theorem 1.3. Let X be a set of2n variables and let f G F[X] be a full max-rank polynomial. 
Let $ be any {s,d) -product-sparse formula of size n^^°^", for a constant e. If sd = o(n^/^), 
then f cannot be computed by $. 

We also generalize the above theorem to the case of preprocessesed product-sparse for- 
mulas. A preprocessed product-sparse formula can be viewed as obtained from a product- 
sparse formula by applying a preprocessing step in which each occurrence of input variables 
is replaced by a non-constant univariate polynomial. Different instances of the same input 
variable are allowed to be replaced by different univariate polynomials. 

As our fourth result, we define partitioned arithmetic branching programs which are gen- 
eralizations of ordered ABPs. While ordered ABP can only compute multilinear polynomials, 
partitioned ABP is a non-multilinear model, thus can compute non- multilinear polynomi- 
als too. Moreover, exponential lower bounds are known for ordered arithmetic branching 
programs [7]. We prove an exponential lower bound for partitioned ABPs. 

Theorem 1.4. Let X be a set of2n variables and ¥ be a field. For any full max-rank homo- 
geneous polynomial f of degree n over X and F, the size of any partitioned ABP computing 
f must be 2^(") . 

The rest of the paper is organized as follows. In section [2] we describe formally some 
of the preliminary definitions and notations. In Section [3] we define the main parameter of 
our lower bounds - the polynomial coefficient matrix and prove the required properties with 
respect to arithmetic operations. Section H] presents the lower bound result against depth-3 
homogeneous circuits for computing iterated matrix multiplication. In section [5], we present 
an exponential lower bound for EIIE circuits of bounded product dimension. In section Ej 
we present super-polynomial lower bounds on preprocessed product-sparse formulas. In 
section [7] we prove exponential lower bounds on partitioned arithmetic branching programs. 

2 Preliminaries 

In this section, we formally define the models we study. For more detailed account of the 
model and the results we refer the reader to the survey [T7] . 

An arithmetic circuit $ over the field F and the set of variables X = {xi, X2, . . . , a;„} is a 
directed acyclic graph G = (V, E). The vertices of G with in-degree are called input gates 
and are labelled by variables in X or constants from the field F. The vertices of G with 
out-degree are called output gates. Every internal vertex is either a plus gate or a product 
gate. We will be working with arithmetic circuits with a single output gate and fan-in of 
every vertex being at most two. The polynomial computed by the arithmetic circuit is the 
polynomial associated with the output gate which is defined inductively from the polynomials 
associated with the nodes feeding into it and the operation at the output gate. The size of $ 
is defined to be the number of gates in $. For a vertex v E V, we denote the set of variables 
that occur in the subgraph rooted at v hj Xy. 

We consider depth restricted circuits. A EES circuit is a levelled depth-3 circuit with a 
plus gate at the top, multiplication gates at the middle level and plus gates at the bottom 
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level. The fan-in of the top plus gate is referred to as top fan-in. A EHE circuit is said to be 
homogeneous if the plus gate at the bottom level compute homogeneous linear forms only. 

An important restricted model of arithmetic circuits is multilinear circuits. A polynomial 
/ G ¥[X] is called multilinear if the degree of every variable in / is at most one. An arithmetic 
circuit is called multilinear if the polynomial computed at every gate is multilinear. An 
arithmetic circuit is called syntactic multilinear if for every product gate v with children vi 
and V2, Xy-^ fl X^.^ = (p. 

An arithmetic circuit is called an arithmetic formula if the underlying undirected graph 
is acyclic i.e. fan-out of every vertex is one. An arithmetic circuit is called skew if for every 
product gate, at least one of its children is an input gate. A circuit is called weakly skew if 
for every product gate, at least one of its incoming edges is a cut-edge. 

Let $ be a formula defined over the set of variables X and a field F. For a product gate 
V in $ with children Vi and V2, let us define the following properties: 

Disjoint v is said to be disjoint if Xy^ fl X^^ — (f). 

Sparse v is said to be s-sparse if the number of monomials in the polynomial computed by 
at least one of its input gates is at most 2*. 

Also, for a node v in $, let us define the product-sparse depth of v to be equal to the 
maximum number of non-disjoint product gates in any path from a leaf to v. 

Definition 2.1. A formula is said to he a {s, d) -product- sparse if every product gate v is 
either disjoint or s-sparse, where d is the product-sparse depth of the root node. 

Clearly, any syntactic multilinear formula is a (s, 0)-product-sparse formula for any s. 
Also, a skew formula is a (0, (i)-product-sparse formula where d is at most the height of the 
formula. Thus, proving lower bounds for product-sparse formulas will be a strengthening of 
known results. We also define an extension of the above class of formulas. 

Definition 2.2. A preprocessed product- sparse formula is a product- sparse formula in which 
each input gate which is labelled by an input variable ( say, Xi ) is replaced by a gate labelled by 
a non-constant univariate polynomial T{xi) in the same variable. The size of a preprocessed 
product-sparse formula is defined to be the size of underlying product- sparse formula. 

An arithmetic branching program (ABP) B over a field F and a set of variables X is 
defined as a 4-tuple (G, w, s, t) where G — (V, E) is a directed acychc graph in which V can 
be partitioned into levels Lq, Li, . . . , such that Lq = {s} and Lj = {t}. The edges in E 
can only go between two consecutive levels. The weight function w : E ^ X U¥ assigns 
variables or constants from the field to the edges of G. For a path p in G, we extend the 
weight function by w{p) = Yle&p'^i^)- ^'^^ ^'.^ ^ denote by Pij the collection 

of all paths from i to j in G. Every vertex v in B computes a polynomial which is given by 
^p€Psv ''^(p)- "^^^ polynomial / computed by B is defined to be the polynomial computed 
at the sink t i.e. / = Z^pgp^ ^ '"^ (p) ■ The size of B is defined to be \V\. The depth of B is 
defined to be d. 
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For any i,j e V, let us denote by Xij the set of variables that occur in paths Pij and 

denote by fij the polynomial X^pgp w{p). A homogeneous arithmetic branching program 
is an ABP B in which the weight function w assigns linear homogeneous forms to the edges 
of B. Clearly, the degree of the homogeneous polynomial computed by B is equal to the 
depth of B. 

Definition 2.3. Let B = {G,w,s,t) be a homogeneous ABP over a field F and set of 
variables X = {xi, X2, ■ ■ ■ , X2n}- B is said to be t: -partitioned for a permutation tt : [2n] 
[2n] if there exists an i — 2an for some constant a such that the following condition is 
satisfied, \/v & : 

• Either, Xs,v Q {^^ttCi) , a^7r(2) , ■ ■ ■ , XT,(n)} and \Xy^t\ <2n{l- a). 

• Or, X^^t ^ {a;^(n+i),a;^{n+2), • • • ,a;^(2n)} (^^d \Xs,u\ < 2n(l - a) 

We say that B is partitioned with respect to the level L^. B is said to be a partitioned ABP 
if it is Ti -partitioned for some tt : [2n] [2n]. 



3 The Polynomial Coefficient Matrix 8^ Properties 

In this section, we introduce the main tool used in the paper and prove its properties. Let 
Y = {yi, 1/2, ... , ym} and Z = {zi, Z2, ■ ■ ■ , Zm} be two sets of variables. Let / G W[Y, Z] be a 
multilinear polynomial over the field F and the variables YUZ. Define L/ to be the 2™ x 2™ 
partial derivatives matrix as follows: for monic multilinear monomials p e F[F],g e F[Z], 
define Lf{p, q) to be the coefficient of the monomial pq in /. Let us denote the rank of 
by rank(Lj). We extend the partial derivatives matrix to non- multilinear polynomials. 

Definition 3.1 (Polynomial Coefficient Matrix). Let f G F[F, Z] be a polynomial over the 
field F and the variables Y U Z. Define Mf to be the 2"* x 2"* polynomial coefficient matrix 
with each entry from the ring ¥[Y, Z] defined as follows. For monic multilinear monomials 
p and q in the set of variables Y and Z respectively, Mf{p, q) = G if and only if f can be 
uniquely written as f = pq{G) + Q, where G.Q E F[Y, Z] such that G does not contain any 
variable other than those present in p and q, Q does not have any monomial m which is 
divisible by pq and which contains only variables that are present in p and q. 

For example, if / = yiZi + ylzi + yiZiZ2 + Zi then Mf{yi, zi) = 1 + yi. Observe that we can 
write, 

f ^^Mf(p,q)pq . 

Also observe that for a multilinear polynomial / G F[y, Z], the polynomial coefficient 
matrix Mf is same as the partial derivatives matrix Lf. For any substitution function 
5 : F U Z ^ F, let us denote by Mf\s the matrix obtained by substituting each variable to 
the field element as given by S at each entry in Mf. We define max-rank of Mf as follows: 

max-rank(M/) = max IrankfMf I <?)} 

■' S:YUZ-^V ■' 
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The following propositions bound the max-rank of the polynomial coefficient matrix. 

Proposition 3.2. Let f G ¥[Y, Z] be a polynomial over the field ¥ and the sets of variables 
Y' CY and Z' C Z. Let a = min{|y'|, \Z'\}. Then, max-rank(M/) < 2«. 

Proof. In the polynomial coefficient matrix Mf, the number of non-zero rows or non-zero 
columns will be at most 2". Thus, rank of Mf for any substitution would be at most 2". 
Hence, max-rank(Mj^) < 2". □ 

Proposition 3.3. Let f,gE ¥[Y, Z] be two polynomials. Then, 

max-rank (Mj+g) < max-rank(Mj) -|- max-rank(M(,). 

Proof. It is easy to observe that Mf^g = Mf + Mg. Let max-rank (Mj+g) = rank{Mf^g\s) 
for some substitution S. Then, 

max-rank(Mj_|_g) = Tank{Mf+g\s) 

= Tank{Mf\s + Mg\s) 

< rank(M/|s) +rank(Mg|s) 

< max-rank(M^) -|- max-rank(Mg). 

□ 

Proposition 3.4. Let Yi, Y2 CY and Zi, Z^^Z such that Fi n F2 = and Zi fl Z2 = 4>. 
Let f e F[yi, Zi] and g e ¥[Y2, Z2]. Then, 

max-rank(M/g) = max-rank(M/) • max-rank(Mp). 

Proof We think of Mf as a 2\^^\ x 2l^il matrix and Mg as a 2l^2| x 2l^2| matrix as all the other 
entries are zero. Similarly, we can think of Mfg matrix. Since / and 

g arc defined over disjoint set of variables, we have Mfg = Mf (E) Mg where (8) denotes the 
tensor product of two matrices. Let max-rank(M/g) = iaiik{Mfg\s) for some substitution S. 
Then, 

max-rank(Mjg) = rank(Mjg|5) 

= Tankl{Mf0 Mg)\s) < rank(M^|s ® 

< rank(M/|5) • rank(Mg|5) < max-rank(M/) ■ max-rank(Mp). 

Similarly, max-rank(Mjg) > max-rank(Mj) • max-rank (Mg). □ 

Proposition 3.5. Let f e W[Y,Z] and g e ¥[Y] or g e ¥[Z]. Then, max-rank(M/g) < 
max-rank(My) . 

Proof. Without loss of generality, we assume that g e ¥[Y]. The case when g e ¥[Z] 
will follow similarly. For a subset C y, we denote the monomial Ilyi^sy by y^. Let us 
analyze the case when g — y^. Consider a row of Mfg indexed by the multilinear monomial 
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p in the set of variables Y . If p is not divisible by y , then all the entries in this row 
will be zero. Otherwise, for any multilinear monomial q in the variables Z, we can write, 
MySf = Es^csy'^^^'Mfip/y'^q) . Thus, rows in MyS j are a linear combination of rows in 
Mf. Similarly, we can show that for any monomial m in the variables Y, rows in M^/ are a 
linear combination of rows in Mf. 

Now consider any g = Ylie[r] ^ ^[Y] where r is the number of monomials in g and 
each nii is a distinct monomial. Thus, Mfg = ^^^^j.] ^ruif- Thus, each row in Mjg is a linear 
combination of rows in Mf. Hence, max-rank(Mjg) < max-rank(Mj). □ 

Corollary 3.6. Let f,g & ¥[Y,Z]. If g is a linear form, then max-rank(M/g) < 2 • 
max-rank(M/). 

Proof. Since g is a linear form in the variables Y U Z, g can be expressed as g = gi + g2 
where gi G ¥[Y] and g2 G ¥[Z] and the proof follows. □ 

Corollary 3.7. Let f,g E ¥[Y,Z]. If g can be expressed as di^i where gi G ¥\Y] and 

je[r] 

hi G ¥[Z], then max-rank(M/g) < r ■ max-rank(Mj). 

Proof. Since Mjg = Yl ^fgthi, using Proposition 13.51 completes the proof. □ 

ie[r] 

Corollary 3.8. Let f,g E ¥[Y,Z]. If g has r monomials, then max-rank(M/g) < r • 
max-rank(Mj) 

Proof. Each monomial rrii of g can be written as gihi such that gi is a monomial in the 
variables Y and hi is a monomial in the variables Z. Thus, the proof follows using above 
corollary. □ 

Full Rank Polynomials 

Let X = {xi, ■ ■ ■ , X2n}, Y = {yi, ■ ■ ■ , ?/„} and Z = {zi, ■ ■ ■ , z„} be sets of variables and 
/ G F[X]. / is said to be a full rank polynomial if for any partition A : X ^ Y U Z, 
rank(LjA) = 2", where is the polynomial obtained from / after substituting every variable 
X by A{x). We say that / is a full max-rank polynomial if max-rank(MjA) = 2" for any 
partition A. Observe that any full rank polynomial is also a full max rank polynomial. 
Further more, many full rank polynomials have been studied in the literature [U [TT|, \T2\ . 

4 Lower Bounds against Homogeneous Depth-3 Cir- 
cuits 

We recall the definition of homogeneous EIIE circuits from section [2l The polynomial 

k 

computed by a SIIS circuit with top fan-in k can be represented as ^ -Pj, where Pi = 

i=l 
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deg(P,) 

Yl h,j, each lij is a linear from and deg{Pi) is the fan- in of the i^^ muhiphcation gate at 
the middle level. 

Let $ be a homogeneous SITS circuit defined over the set of variables X and over a field 
F computing a homogeneous polynomial /. Let us denote the polynomial coefficient matrix 
of the polynomial computed at the top plus gate of $ by M<j,. For a partition A : X ^ YUZ, 
let us denote by the circuit obtained after replacing every variable x by A{x). We prove 
the following upper bound on the max-rank(M$A). 

Lemma 4.1. Let ^ be a homogeneous SIIS circuit as defined above. Let the degree of f be 
equal to d. Then, for any partition A : X ^ Y U Z , max-rank(M$A) < k ■ 2'^. 

k deg{Pi) 

Proof. From the definition it is clear that / can be written as: / = ^ Pi where Pi = Yl 

i=i j=i 

each lij is a homogeneous linear form. Let us denote by ifj and Pf" the polynomials obtained 
after substitution of x by A{x) in the polynomials and Pi respectively. 

Since each is a homogeneous linear form, a multiplication gate Pi computes a homo- 
geneous polynomial of degree deg{Pi). Thus if deg{Pi) ^ d then the multiplication gate Pi 
does not contribute any monomial in the output polynomial /. Hence, it can be assumed 
without loss of generality that deg{Pi) = d for all i G [k]. 

Since lij is a homogeneous linear form, max-rank(M;A ) < 2. Thus, using Corollary 

\3Myi G [k] : max-rank(MpA) < 2'^. Hence, using Proposition 13.31 max-rank(MjA) < 
Y.ie[k] max-rank(MpA) < k ■ 2'^. □ 

In [9], it was proved that any homogeneous SHE circuit for multiplying d n x n matrices 
requires Q{n'^~^ / d\) size. We prove a better lower bound using our techniques. To consider a 
single output polynomial, we will concentrate on the (1, 1)*'^ entry of the product. Formally, 
let X^jX"^, . . . ,X'^ be disjoint sets of variables of size each, with X = Ui^[d]X^. The 
variables in X* will be denoted by x* j^, for j, k E [n]. We will be looking at the problem of 
multiplying d n x n matrices A^,A'^,...,A'^ where {j, ky^ entry of matrix A\ denoted by 
A'^jf^, is defined to be equal x*;, for all i G [d] and j, k E [n]. The output polynomial, that we 
are interested in, is the (1, 1)*'* entry of Hieid] denoted by /. / is clearly a homogeneous 
multilinear polynomial of degree d. Moreover, any monomial in / contains one variable each 
from the sets X\ X^, . . . , X'^. 

We first prove an important lemma below. We also provide an alternative induction 
based proof for the below lemma in the Appendix Rl 

Lemma 4.2. For the polynomial f as defined above, there exists a bijective partition B : 
X U Z such that max-rank(MjB) = n'^'^. 

Proof. We fix some notations first. For i < j, let us denote the set {i,i + by 
[i,j]. Let us also denote the pair {{k,i), {k, j)) by Cijk for any i,j,k. Construct a di- 
rected graph G(y,E) on the set of vertices V = [0,d] x [l,n] and consisting of edges 
E = {cijk I G [0,(i— G [l,""-]}. Note that the edges Cijk and Cjik are two distinct 
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edges for fixed values of i,j,k when i ^ j. Let us also define a weight function w : E X 
such that w{eijk) = x'l^^. 

It is easy to observe that the above graph encodes the matrices Ai, . . . , A^. The 
weights on the edges are the variables in the matrices. For example, a variable x\^^ in 
the matrix Ak+i is the weight of the edge eijk- Let us denote the set of paths in G from 
the vertex (0, 1) to the vertex (ci, 1) by V. Let us extend the weight function and define 
'^{p) = neep^(^) ^'^y p Since, all paths in V are of length equal to rf, the weights 
corresponding to each of these paths are monomials of degree d. 

Let us define the partition B : X VJ Z a.s follows: all the variables in odd numbered 
matrices are assigned variables in Y and all the variables in even numbered matrices are 
assigned variables in Z. Let us denote the variable assigned by B to by ylj^^ and the 

variable assigned to xf^ by zf^. 

It follows from the matrix multiplication properties that for any path p G P, the monomial 
w{p) is a monomial in the output polynomial. Each such path is uniquely specified once we 
specify the odd steps in the path. Now, specifying odd steps in the path corresponds to 
specifying a variable from each of the odd numbered matrices. To count number of such 
ways, let us first consider the case when d is even. There are d/2 odd numbered matrices 
and we have ways to choose a variable from each of these d/2 matrices except for the first 
matrix for which we can only choose a variable from the 1** row since our output polynomial 
is the (1, 1)*'^ entry. Thus, there are n'^~^ number of ways to specify one variable each from 
the odd numbered matrices, the number of such paths is also n'^~^. We get the same count 
for the case when d is odd using the similar argument. Since once the odd steps are chosen, 
there is only one way to choose the even steps, all these n'^~^ monomials give rise to non-zero 
entries in different rows and columns in the matrix Mjb. Hence, the matrix is an identity 
block of dimension n'^~^ upto a permutation of rows and columns and thus it has rank equal 
to n'^-^. □ 

Theorem 4.3. Any homogeneous SHE circuit for computing the product of d nxn matrices 
requires Vt{n'^~^ / 2'^) size. 

Proof. Let $ be a homogeneous depth-3 circuit computing /. Then, using Lemma [4.H for 
any partition A, max-rank(MjA) < k ■2'^. From Lemma |4.2[ we know that there exists a 
partition B such that max-ran k{MfB) = n'^-\ Hence, k > n'^-^/2'^. □ 

It is worth noting that there exists a depth-2 circuit of size n'^~^ computing IMM polyno- 
mial. As observed in Lemma [4. 2[ there are n'^~^ monomials in the IMM polynomial. Hence, 
the sum of monomials representation for IMM will have top fan- in equal to n'^~^. We re- 
mark that when the number of matrices is a constant, the upper and lower bounds for IMM 
polynomial match. 
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5 Lower Bounds against Depth-3 Circuits of Bounded 
Product Dimension 



If a depth-3 circuit is not homogeneous, the fan-in of a product gate can be arbitrarily larger 
than the degree of the polynomial being computed. Hence the techniques in the previous 
section fails to give non-trivial circuit size lower bounds. In this section, we study depth-3 
circuits with bounded product dimension - where the affine forms feeding into every product 
gate are from a linear vector space of small dimension and prove exponential size lower 
bounds for such circuits. 

We will first prove an upper bound on the maxrank of the polynomial coefficient matrix 
for the polynomial computed by a depth-3 circuit of product dimension r, parameterized by 
r. Let C be a SIIS circuit of product dimension r and top fan in k. Let be the product 
gates in C for j G [k], given by = II^^^^L^. Without loss of generality, let us assume 
that the vectors L{, L^, . . . ,Ll form a basis for the span of {L{, L^, . . . , L^}. Let be the 
homogeneous part of Ll for each i. So, clearly the set {ll}ifz[r'] spans the set {/^}jg[s], where 
r' < r. To simplify the notation, we will refer to r' as r. In the following presentation, we will 
always use d to refer to the degree of the homogeneous polynomial computed by the circuit 
under consideration. Now, let us express each as a linear combination of {ll}i<^[r]. Let us 
now expand the product P^ into a sum of product of homogeneous linear forms coming from 
{ll}i£r- Let Pj be the slice of P^ of degree exactly d, for each j G [k]. We now have the 
following observation. 

Observation 5.1. Let = Ejg[fc]P^. If C computes a homogeneous polynomial of degree d, 
then Cd computes the same polynomial. 

Proof. The proof follows from the fact that since C computes a homogeneous polynomial 
of degree d, the monomials for degree other than d cancel each other across the different 
product gates. □ 

We now look at each product in Pj, which is a sum of products. Each such product 
is a product of homogeneous linear forms from {ll}i(z[r] of degree exactly d. To simplify it 
further, we will use the following lemma. 

Lemma 5.2. (ITE^ ) Any monomial of degree d can be written as a sum of d^^ power of some 
2'^ linear forms. Further, each of the 2°^ linear forms in the expression corresponds to Saie^x 
for a subset S of [d]. 

By applying this lemma to each product term in the sum of product representation of 
Pj, we obtain the following: 

Lemma 5.3. // Pj = SjllJ^^^^Z^,^ where aiu G [r], then Pj = S^^j^CgLg'^ for some homoge- 
neous linear forms Lq, constants Cq and v < ('^^^) ■ 
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Proof. Consider any product term in the sum of products expansion Pj as described, say 
S = Hu^i^a. . From Lemma [521 we know that 5* can be written as S* = T,f^-^^L/, where for 
every subset U of [d], there is a /3 G [2*^] such that = Sug(//^.^. In general, each Lt can 
be written as Lt = Sig[r]7j/^ for non-negative integers 7^ satisfying Ejg[r]7j < d. Now, each 
of the product terms in Pj can be expanded in a similar fashion into d*'^ powers of linear 
forms, each from the set {Ejg[r]7jZ^ : 7^ G A Sjg[r]7i < d}. The number of distinct such 
linear forms is at most {'^^^)- Hence, the lemma follows. □ 

We now bound the maxrank of the power of a homogeneous linear form which in turn 
will give us a bound for Pj due to the subadditivity of maxrank. 

Lemma 5.4. Given a linear form I and any positive integer t, the maxrank of is at most 
t + 1 for any partition of the set X of variables into Y and Z . 

Proof. Partition the linear form / into two parts, / = ly + lz, where ly consists of all variables 
in / from the set Y and Iz consists of the variables which come from the set Z. By the 
binomial theorem, = S*^Q(*)Zy/*~*. Now, ly is a polynomial just in Y variables and hence 
its maxrank can be bounded above by 1, and multiplication by does not increase the 
maxrank any further, by proposition 13.51 Hence, the maxrank of each term in the sum is at 
most 1 and there are at most t + 1 terms, so, by using the subadditivity of maxrank, we get 
an upper bound of t + 1 on the maxrank of the sum. □ 

Now we are all set to give an upper bound on the maxrank of Pj. 

Lemma 5.5. The max rank of P^ is at most {d + for any partition of the set X of 

variables into Y and Z . 

Proof. The proof follows from Lemma [531 Lemma [5741 and the subadditivity of max rank. □ 
Now we are ready to prove the theorem. 

Theorem 5.6. There is an explicit polynomial in n variables X and degree at most | for 
which any SHE circuit C of product dimension at most requires size 2^("\ 

Proof. We describe the explicit polynomial Q{X) first. Fix an equal sized partition A ol X 
into Y and Z. Order all subsets of Y and Z of size exactly | in any order, say Si, S2, ■ ■ ■ ,3^ 

l). Let us define the polynomial Q (Y, Z) for the partition 

4 

A as follows: 

Q^(r,Z) = Sr=iH,e5,H,eT,2/^ 

We obtain the polynomial Q{X) by replacing variables in Y and Z in Q'^iY, Z) by A^^{Y) 
and A^^{Z) respectively. The polynomial Q{X) is homogeneous and of degree |. 

Now we prove the size lower bound. The polynomial coefficient matrix of Q with respect 

n 

to the partition Y and Z is simply the diagonal submatrix, and the rank is at-least ^ Since 

n 

the circuit computes the polynomial, the top fan in k should be at least jj+Tj^d+r)^ ^"^^ ~ t' 
and product dimension j^, we have a lower bound of 2^^", for a constant c > 0. □ 
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An Impossibility result: Consider the trivial depth-2 circuit for any polynomial, where 
each monomial is computed by the product gate. Viewing this as a depth-3 circuit, the total 
dimension of the circuit is bounded above by n, since there are only n variables. Can we have 
a circuit with a smaller total dimension r computing the same polynomial? We show that 
this is not always possible if r = a.n for a sufficiently small constant a < 1. In particular, 
we show that even for r = -^i they cannot compute the polynomial that we constructed in 
the proof of theorem 15.61 irrespective of the size of the circuit. As a first step, using ideas 
developed in the previous subsection, we prove the following upper bound for maxrank of 
such circuits. 

Lemma 5.7. If the total dimension of a EIIE circuit is r, then the maxrank of the polynomial 
computed by the circuit is hounded above by {^^^^{d+ 1). 

Proof. Observe that if the span of all the affine forms occurring in a depth-3 SHE circuit 
is r (spanned by the basis Li, L2, . . . , Lr), then each of the product gates in the circuit can 
be decomposed into sum of power of homogeneous linear forms as in Lemma 15.31 Moreover, 
each of these homogeneous linear forms will be of the form SjCtj/j, where G A EjOj < d 
and li is the homogeneous part of Lj for each i in [r]. Consequently, the maxrank for the 
circuit is bounded by {^^^^{d+ 1) by Lemma [5.41 and the subadditivity of max rank. □ 

Thus, a SnS circuit of total dimension bounded by r, can compute the polynomial Q 
described in the proof of I5.6[ only if 



spective of the number of gates they use. 

6 Lower Bounds against Product-sparse Formulas 

Let Y = {yi, ?/2, • • • , ym} and Z = {zi, Z2, . . . , Zm}- Let $ be an arithmetic circuit defined 
over the field F and the variables Y U Z. For a node v, let us denote by $^ the sub-circuit 
rooted at v, and denote by Y^ and Z^, the set of variables in Y and Z that appear in $^ 
respectively. Let us define, a{v) = min{|Fi;|, \Z^\} and b{v) = {\Yy\ + \Zy\)/2. We say that a 
node V is fc-unbalanced if b{v) — a{v) > k. Let 7 be a simple path from a leaf to the node 
V. We say that 7 is fc-unbalanced if it contains at least one /c-unbalanced node. We say that 

7 is central if for every u,Ui on the path 7 such that there is an edge from ui to u in $, 
b{u) < 2b{ui). V is said to be /c-weak if every central path that reaches v is fc-unbalanced. 

We prove that if v is fc-weak then the max-rank of the matrix can be bounded. The 
proof goes via induction on |$^| and follows the same outline as that of [12]. It only differs 
in the case of non-disjoint product gates which we include in full detail below. The proofs 
of the rest of cases is given in the appendix [Bl 




This in turn implies that for r < f^, such circuits cannot compute the polynomial Q irre- 
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Lemma 6.1. Let ^ be a (s, d) -product- sparse formula over the set of variables {yi, . . . , ym} 
and {zi, . . . , Zm}, and let v be a node in $. Denote the product- sparse depth of v by d{v). If 
V IS k-weak, then, max-rank(M^) < 2'<''^ ■ \^v\ ■ 2^^"^-^/"^ . 

Proof. Consider the case when f is a s-sparse product gate with children vi and V2 and v. 
Without loss of generality it can be assumed that v is not disjoint. 

Let us suppose that the product-sparse depth of v is d. Without loss of generality, assume 
that V2 computes a sparse polynomial having at most 2** number of monomials. Since v is 
not disjoint, product-sparse depth of vi is at most d — 1. Thus using Corollary 13. 8[ 

max-rank(My) < 2* ■ max-rank(M„J (1) 

Consider the following cases based on whether h{v) < 2b{vi) or not. 

If b{v) < 2b{vi), then Vi is also k-weak. Therefore, by induction hypothesis, 

max-rank(M^J < 2^('^-i) ■ ■ 2''('^^)-'^/2 < 2'^'^^^^ ■ • 2^'^"'^'^/^ . 

Thus, using Equation [H max-rank(M„) < 2"^ ■ |$^| ■ 2^^'''^~^'^ . If b{v) > 2b{vi), then b{vi) < 
b{v)/2 < b{v) — k/2 since b{v) > k. Therefore using Proposition 13. 2[ max-rank(M„J < 
2a(vi) < 2''M < 2M^)-fc/2, Therefore, max-rank(M^) < 2' ■ 2^(^)-'=/2 < 2"'=' ■ |$^| ■ 2^^"^-''/^ . □ 

Because of previous lemma, to prove that a full max-rank polynomial cannot be computed 
by any (s, (i)-product-sparse formula of polynomial size, we only need to show that there 
exists a partition that makes the formula fc-weak with suitable values of s, d and k. 

In [11], it was proved that for syntactic multilinear formulas of size at most ?t,^^°^", where 
e is a small enough universal constant, there exists such a partition that makes the formula 
fc-weak for k = n^^^. We observe that this lemma also holds for product-sparse formulas, 
the proof given in [TT] is not specific to just syntactic multilinear formulas and holds for any 
arithmetic formula. We state the lemma again for the case of product-sparse formulas. 

Lemma 6.2. Let n = 2m. Let ^ be a (s, d) -product- sparse formula over the set of variables 
X = {xi, . . . ,Xn}, such that every variable in X appears in $, and such that |$| < n^^°^", 
where e is a small enough universal constant. Let A be a random partition of the variables in 
X into {yi, . . . , y^n} U {zi, . . . , Zm}- Then with probability of at least 1 —n^^(^°sn) ^/^g formula 
is k-weak, for k = n^^^ . 

With above lemma, the following theorem becomes obvious. 

Theorem 6.3. Let X be a set of2n variables and let f G F[X] be a full max-rank polynomial. 
Let $ be any {s, d) -product- sparse formula of size n'^^°^"', where e is the same constant for 
which Lemma \6. 2\ holds. If sd = o{n^^^), then f cannot be computed by $. 

Proof. Assume for a contradiction that $ computes /. Using Lemma 16. 2^ for a random 
partition A, with probability of at least 1 — n~^^^°^"'\ the formula is A;- weak for k = n^^^. 
Hence, using Lemma |6TT| max-rank(M,j,A) < 2*'^- 1$"^! .2"~'^'/^ < 2". Since / is a full max-rank 
polynomial, it cannot be computed by $. □ 



14 



Preprocessed Product-sparse Formulas 

To prove the results about preprocessed product-sparse formulas, we observe first that the 
Lemma [6. II also holds for preprocessed product-sparse formulas. 

Lemma 6.4. Let ^ be a preprocessed {s,d) -product-sparse formula over the set of variables 
{yi, . . . ,yn} and {zi, . . . ,Zn}, and let v be a node in $. Ifv is k-weak, then, 

max-rank(M^) < 2'-'^^^^ ■ |$^| ■ 2^'^'"'^-''/^ . 

where d{v) is the product- sparse depth of v. 

Proof. The proof proceed in the same way by induction on |$^|. We only point out the 
differences in the two proofs. Base case will hold similarly as for any univariate polynomial 
T(x), max-rank(MT(2:)) is at most one. In the induction step, only difference will be in Case 
(3) in which f is a product gate obeying s-sparse property. In lemma 16. ![ we obtain the 
following inequality using Corollary ESI 

max-rank(Mt,) < 2* ■ max-rank(Mt,J . 

We obtain the same inequality for preprocessed product-sparse formulas also using Corollary 
13.71 instead of Corollary 13.81 and rest of the proof follows similarly. □ 

We also observe that if a product-sparse formula $ is fc-weak, then any preprocessed 
product-sparse formula obtained from $ by applying a preprocessing step is also k-weak. 

Lemma 6.5. Let ^ be a product- sparse formula over the set of variables Y U Z . Let $' be 
any preprocessed product- sparse formula obtained from $. If ^ is k-weak, then $' is also 
k-weak. 

Proof. For every node v in there is a corresponding node v' in To prove the lemma 
we need to show that every central path in $' is k-weak. Since in the preprocessing step, 
each variable is replaced by a non-constant univariate polynomial in the same variable, we 
know that Y^i = and Z^i = Z^. Thus, a{v') = a{v) and b{v') = b{v). Hence, every central 
path in $' is a central path in $ and vice-versa. Also, v is fc-unbalanced in $ iff v' is also 
fc-unbalanced in Hence, if $ is k-weak then $' is also fc-weak. □ 

By using Lemma 16. 4[ in a very similar way to the proof of Theorem 16. 3[ we get the 
following: 

Theorem 6.6. Let X be a set of2n variables and let f G ¥[X] be a full max-rank polynomial. 
Let $ be any preprocessed {s,d) -product-sparse formula of size n'^^°^", where e is the same 
constant for which Lemma \6. 2\ holds. If sd = o{n^^^), then f cannot be computed by 

Proof. Assume for a contradiction that $ computes /. Using Lemma 16.21 and 16. 5[ for a 
random partition A, with probability of at least 1 — n~^^^°^'^\ the formula is k-weak for 
k = n^/^. Hence, using Lemma [6.4[ 

max-rank(M$A) < 2^^^ ■ |$^| ■ 2"-'=/^ < 2". 

Since / is a full max-rank polynomial, it cannot be computed by $. □ 
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7 Lower Bounds against Partitioned Arithmetic Branch- 
ing Programs 

In the preliminaries section, we defined partitioned arithmetic branching programs which 
are a generahzation of ordered ABPs. While ordered ABP can only compute multilinear 
polynomials, partitioned ABP is a non-multilinear model, thus can compute non-multilinear 
polynomials also. We, then, prove an exponential lower bound for partitioned ABPs. 

By definition, any polynomial computed by a partitioned ABP is homogenous. In [7], 
a full rank homogenous polynomial was constructed. Thus, to prove lower bounds for par- 
titioned ABP, we only need to upper bound the max-rank of the polynomial coefficient 
matrix for any polynomial being computed by a partitioned ABP. Now we prove such an 
upper bound and use it to prove exponential lower bound on the size of partitioned ABPs 
computing any full max-rank homogenous polynomial. 

Theorem 7.1. Let X be a set of 2n variables and ¥ be a field. For any full max-rank ho- 
mogenous polynomial f of degree n over X and ¥, the size of any partitioned ABP computing 
f must be 2^^'^). 

Proof. Let i? be a vr-partitioned ABP computing / for a permutation vr : [2n] — )■ [2n\. Let 
Lq, Li, . . . , Ln be the levels of B. Consider any partition A that assigns all n y- variables to 
x^(^2), • • • , x^(^n)} and all n 2;- variables to a;^(„+2), • • • , x^(^2n)}- Let us denote 

by f^ the polynomial obtained from / after substituting each variable x by A{x). Let B is 
partitioned with respect to the level Li for i = 2an. We can write, f = fst = J2veL^ fs,vfv,t ■ 
Consider a node v & Li. By definition, there are following two cases: 

Case 1: X^^^ C x^(2), . . . , and \X^,t\ < 2n(l - a). Thus, f^^ G ¥[Y]. Hence, 

using Proposition 13. 5[ 

max-rank(M^A^^Aj < max-rank(M^Aj < 2^^^-'^/^ < 2"(^~") 

Case 2: X^^f C {XT^(^n+l):^n{n+2)y ■ ■ ■ :^TT(2n) 

} and |X,,„| < 2n(l - a). Thus, f^^ G ¥[Z]. 

Hence, again using Proposition 13. 5[ 

max-rank(M^A_^^Aj < max-rank(MjAj < 2l^^-"l/2 < 2"(^~°) 

Thus, in any case, max-rank(MjA j-aJ < 2"(^~") for all v G Li. Using Proposition 13. 3[ 
max-rank(MjA) < |Lj| ■2^^^~°'\ Since / is a full max-rank polynomial, we get |Lj| > 2"". □ 
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A An alternative Proof of Lemma 
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We view the product of d matrices as an iterative process using the associative property of 
the matrix multiphcation and prove a more stronger statement than Lemma 14.21 We will 
first prove it for the case when d is an even integer. Let us define a partition B : X ^ YU Z 
as follows: all the variables in odd numbered matrices are assigned variables in Y and all the 
variables in even numbered matrices are assigned variables in Z. Let us denote the variable 
assigned by B to x'^lT'^ by yfi^^ and the variable assigned to x'jl by Zjl. 

Lemma A.l. Let d = 2d' be an even integer. Let us denote the polynomial computed at 
{i,jY^ entry of the product Y[ by fij. The following statements hold true: 

keld] 



1. For any i,j G [n], rank(Mjs) = n'^ ^ . 

2. For any i G [n] and j ^ j' G [n], the set of non-zero columns in M^b and Mjb^ are 
disjoint. 

Proof. We will prove by induction on d'. 

Base Case: d' = 1 i.e. d = 2. In this case, we have two matrices and and the 
partition B assigns all the variables in A^ to Y and all the variables in A"^ to Z. For any i,j, 
= = ^ A\t,k)A\k,j). Thus, = E y}k4r Clearly, rank(M^B) = n. 

Moreover, there are only n non-zero columns in Mjb which are indexed by zlj respectively. 
Thus, for j ^ j' the set of non-zero columns in MfB and MfB are disjoint. 
Induction Step: Let us suppose the lemma holds for (d'— 1). We will prove that it also holds 
for d'. Let d = 2d', Q= fl ^'^ and P = A'^-^A'^. Therefore, 11^'' = QP- Thus, fij = 

ke[d-2] ke[d] 

{QP){i,i)= E A;)-P(A;,j) where P(/c,j) can again be written as ^ A'^~^{k,r)A'^{r, j). 

fce[n] re[n] 

Thus, 

fcG[n] r6[n] 

Let us denote the polynomial Q{i, k) by Qik and ^ (?.^ ?/^~^ z'^^ by Pk- Thus, 

re[?i] 

/if = dfk ytr ^ ^tj = E Pk ■ 

fce[n]re[n] fce[n] 

By induction hypothesis, rank(M^fl) = n'^~^. Thus, MgB^ has a sub-matrix of size n'^~^ x 
TT,'^"^ ^j-^icii is of full rank. Since the variables y'^~^ and z^j do not appear in gff^, the matrix 
Mb d-i d contains the full rank submatrix of M„s at a shifted position in both rows and 

9ik Vkr ^rj ^ik 

columns depending on yf^^ and z^j. Formally, for any monomials p, q in the set of variables 
Y and Z respectively, M b ,d-i {yiZ^P, zf,q) = 1 iff M B{p,q) = 1. Thus, Mp contains 
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n disjoint copies of the full rank sub-matrix of MgB_ such that no two of the copies contain 
any common non-zero row or column. Thus, rank(Mpj.) = n ■ rank(M^s) = n'^^'^. 

By induction hypothesis, we also know that the set of non-zero columns in M b and 
MgB ^ are disjoint for k ^ k'. Thus, for k ^ k , Mp^ and Mp^, do not contain any common 

non-zero column. Hence, 



rank(MjB) = ^ rank(MpJ 



To prove statement 2, it is sufficient to observe that each non-zero entry in MfB is present in 

a column such that the monomial indexing it is divisible by some variable in the j^^ column 
of A'^ and is not divisible by any variable present in other columns of A'^. Thus, for j ^ j', 
the set of non-zero columns in MfB and MfB are disjoint. □ 

■lij J ijl 

For the case when c? is a odd integer, let us denote the polynomial Ilfeeld-i] ^'^(^'i) 
fij and Ukeld]^''i^^j) by 9tj for all ij G [n]. Thus, gij = T^kein] fik^kj ^^ich implies 
dFj — J2ke[n] fikVkj- Thus, MgB contains a copy of Mjs for each k and none of these copies 
have a common non-zero row due to multiplication by y''^. Moreover, we know that MfB 

^ ik 

and MfB do not have any common non-zero columns. Thus, M b contains a copy of the 
full rank sub-matrix of MfB for each k with no common non-zero rows or columns. Hence, 
rank(M r) = V rank(M.s) = n'^'^ . 

ke\n] 



B Complete Proof of Lemma 16.1 



Lemma. Let ^ be a (s, d) -product-sparse formula over the set of variables {yi, . . . , ym} and 
{zi, . . . , Zm}, and let v be a node in $. Denote the product-sparse depth of v by d{y). If v is 
k-weak, then, max-rank(M^) < 2'<''^ ■ |<I>J ■ 2^'^"^'^'^ . 

Proof. We will prove by induction on 

Base Case: f is a leaf node. By definition, the polynomial computed at node v is either 
a constant from the field or a single variable. Thus, max-rank(Mt,) < 1. Thus the lemma 
follows. 

Inductive Step: Let f be a node in $ and assume that the lemma holds for all nodes u in 
$ such that < We consider the following cases: 

1. f is a fc- unbalanced node. 

Thus, a{v) < b{v) — k. Hence, using Proposition 13. 2[ max-rank(Mt,) < 2^^'"^~^ < 
. |$^| .2''('')-^/2. In the rest of the cases, we can assume that v is not ^-unbalanced. 

2. f is a disjoint product gate with children vi and V2 and v. 

Thus, we have Y^^ ft Y^^ = and Z^,^ fl Z^^ = 0. Thus, b{v) = b{vi) + b{v2)- Without 
loss of generality, we can assume that b{v) < 2b{vi). Since v is not /c-unbalanced. 
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every central path that reaches Vi must be fc-unbalanced as otherwise we can extend 
such a path to v that will remain central and not /^-unbalanced. Thus Vi is also 
/c-weak and we have |$t,J < Hence by induction hypothesis, max-rank(Mt,J < 

2'dM.^^^^^.2Hvi)-k/2 _ Using Proposition[321 we have, max-rank(M^J < 2''(^2) < 2'>M . 
Hence by Proposition I3.4[ 

max-rank(Mu) = max-rank(Mt,J ■ max-rank(Mt,2) 

^ 2sd{vi) _ 1^ I _ 2H^'i)+K'"2)-k/2 

3. f is a s-sparse product gate with children vi and V2 and v. Without loss of generality 
it can be assumed that v is not disjoint. 

Let us suppose that the product-sparse depth of v is d. Without loss of generality, 
assume that V2 computes a sparse polynomial having at most 2* number of monomials. 
Since v is not disjoint, product-sparse depth of Vi is at most d — 1. Thus using Corollary 



max-rank(M„) < 2"" ■ max-rank(M„J (2) 

Consider the following cases based on whether b{v) < 2b{vi) or not. 

If b{v) < 2b{vi), then vi is also fc-weak. Therefore, by induction hypothesis, 

max-rank(M^J < 2^('^-i) ■ |$^J ■ 2''(^i)-'=/2 < 2'^"^-^^ ■ \<l>^\ ■ 2''(^)-'=/2 . 

Thus, using Equation El max-rank(M„) < 2""^ ■ ■ 2^('')-^/2 _ jf 5^^) > 2b{vi), 
then b{vi) < b{v)/2 < b{v) — k/2 since b{v) > k. Therefore using Proposition 13. 2[ 
max-rank(M^J < 2*^(^1) < 2''(^i) < 2''(^)-'=/2_ Therefore, max-rank(M^) < 2' ■2^^'"^-''/'^ < 

2sd _ I _ 2^(^)^^/2 

4. f is a plus gate with children vi and V2- 

We know that b{v) < b{vi)+b{v2). Without loss of generality, assume that b{v) < 2b{vi) 
which implies that vi is also fc-weak. Hence by induction hypothesis, 

max-rank(M^J < 2'"^^^''^ ■ ■ 2^(^i)-'=/2 < 2"'^^'') ■ |$^J ■ 2^'^'"'^-^^^ . 

We consider the following cases based on whether b{v) < 2b{v2) or not: 
If b{v) < 2b{v2), then V2 is also A;- weak. Hence by induction hypothesis, 

max-rank(M^J < 2''^^'''^ ■ ■ 2^^''^'>~^/^ < 2'"^^^^ ■ |$^,| ■ 2^('')-^/2_ 

Hence by Proposition 13. 3[ 

max-rank(Mt,) < max-rank(M„J -|- max-rank(My2) 

< 2^'^(^) ■ (|<I>^J + |<I>„J) ■ 2''(^)-'=/2 

< 2*'^*^^) ■ |$^| ■ 2^('')~'^/^ 
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If b{v) > 2b{v2), then b{v2) < b{v)/2 < b{v) — k/2 since b{v) > k. Hence by propo- 
sition ESI max-rank(M^J < 2"(^2) < 2''M < 2^(^)-'=/2 < 2^"'(^) . 2^(^)~^/^ . Hence by 
Proposition 13. 3[ 

max-rank(M^) < 2^<^(^) ■ + 1) ■ 2''^"^-''/^ < 2"^^''^ ■ |$^| ■ 2*('')-^/l 

□ 
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